Papers with transfer learning baselines
Dynamically Updating Event Representations for Temporal Relation Classification with Multi-category Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models with independent classifiers for each TLINK category hinder from using the whole data. |
| Approach: | They propose a temporal relation classification model that manages dynamic event representations across multiple TLINKs using multi-task learning to leverage the full size of data. |
| Outcome: | The proposed model outperforms state-of-the-art models and two strong transfer learning baselines on English and Japanese data. |
A Novel Two-step Fine-tuning Framework for Transfer Learning in Low-Resource Neural Machine Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing transfer learning methods for neural machine translation use a well-trained translation model to initialize a child model with corresponding datasets. |
| Approach: | They propose a two-step fine-tuning framework for transfer learning in low-resource neural machine translation that adjusts the parent model to fit the child language by using the child source data. |
| Outcome: | The proposed framework improves on five low-resource translations on high-resolution languages. |
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions (N19-1)
Copied to clipboard
| Challenge: | In this paper we build a reading comprehension dataset of yes/no questions that are naturally occurring . they often query for complex, non-factoid information, and require difficult entailment-like inference to solve. |
| Approach: | They build a reading comprehension dataset of yes/no questions that are naturally occurring . they find they are unexpectedly challenging and require difficult inferences to solve . |
| Outcome: | The proposed method achieves 80.4% accuracy compared to 90% accuracy of human annotators and 62% majority-baseline. |
Adaptive Multi-Task Transfer Learning for Chinese Word Segmentation in Medical Text (C18-1)
Copied to clipboard
| Challenge: | Chinese word segmentation (CWS) tools face a performance drop when dealing with domain text . domain-specific CWS requires extremely high annotation cost due to ambiguity caused by domain terms and writing style . |
| Approach: | They propose to exploit domain-invariant knowledge from high resource to low resource domains to build Chinese word segmentation models. |
| Outcome: | The proposed model achieves higher accuracy than single-task CWS and other transfer learning baselines . the model is based on domain-invariant knowledge from high resource to low resource domains based in the biomedical domain . |
Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on colexification patterns relies on annotated word lists, limiting scalability and usefulness in NLP. |
| Approach: | They propose two methods to train multilingual graphs from colexification patterns using an unannotated parallel corpus. |
| Outcome: | The proposed methods achieve high recall on CLICS and transfer learning in multilingual graphs. |
ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing transfer learning methods for low-resource NMT are static, which simply transfer knowledge from a parent model to a child model once via parameter initialization. |
| Approach: | They propose a transfer learning method that can continuously transfer knowledge from the parent model during the training of the child model. |
| Outcome: | The proposed method can transfer knowledge from the parent model to the child model during the training of the child. |